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FOREWORD 


This  paper  was  presented  at  the  Second  Congress  on  the  Information 
System  Sciences;  It  appears  in  INFORMATION  SYSTEM  SCIENCES:  PROCEEDINGS 
OF  THE  SECOND  CONGRESS,  published  by  Spartan  Books,  Incorporated  (Washington, 

D.C.),  1965. 
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ABSTRACT 


Decision  theory  is  the  contemporary  manifestation  of  the  mathematics 
of  the  decision  process  and  thus  can  be  viewed  as  a  primary  aid  to  the 
human  decision  process.  The  costs  and  gains  of  applying  the  concepts  and 
algorithms  of  decision  theory  are  considered  in  some  detail.  A  man/computer 
system  is  described  which  is  designed  to  make  the  concepts  and  algorithms 
of  decision  theory  available  to  a  decision  maker  at  a  greatly  reduced  personal 
cost.  This  is  achieved,  in  large  part,  by  significantly  reducing  the  special 
knowledge  required  of  the  decision  maker.  Thus,  the  decision  maker  needs 
no  knowledge  of  computer  programming  and  a  minimal  knowledge  of  decision 
theory  and  mathematics  in  order  to  begin  using  the  system  in  his  day-to-day 
decision  making  activities. 
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CORTEX:  A  Computer-Based  System  for  Aiding  Decision  Making 


Emir  H.  Shuford,  Jr. 


In  a  broadly  conceived,  and  somewhat  trivial,  sense  most  applications 
of  computers  can  be  interpreted  as  attempts  to  improve  the  quality  of 
decisions.  Information  systems  for  the  military  as  well  as  information 
retrieval  systems  for  the  scientist  and  engineer  can  be  viewed  as  attempts 
to  improve  the  quality  of  decisions  by  providing  an  increased  amount  of 
timely  information  to  the  decision  maker.  Computer-controlled  production 
processes  and  accounting  procedures  can  be  seen  as  attempts  to  improve 
the  effectiveness  of  decisions  by  providing  for  the  reliable  and  timely 
realization  of  plans  previously  selected  by  a  decision  maker.  Thus,  computers 
are  used  as  aids  both  in  the  information  gathering  stage  and  in  the  operational 
stage  of  decision  making.  In  addition,  computers  are  used  as  aids  to  the 
decision  process  itself  and  it  is  in  this  sense  that  I  want  to  discuss 
aiding  decision  making. 

An  individual,  e.g.,  a  military  commander  or  staff  officer,  a  business 
manager,  a  design  engineer,  or  a  scientist,  has  a  problem  to  solve  or  a  plan 
to  prepare.  How  can  he  be  helped  in  the  actual  performance  of  the  decision 
process  itself?  Are  there  any  mathematics  available  that  might  be  used  to 
give  logical  consistency  to  his  decisions  and  to  aid  his  understanding  of  the 
nature  of  the  decision  problem?  What  role  might  a  computer  be  expected  to 
perform  in  this  process?  These  are  the  questions  that  T  would  like  to  consider 


in  some  detail. 


The  Mathematics  of  the  Decision  Process. 


Again  in  a  broadly  conceived,  but  not  trivial,  sense  the  use  of  applied 
mathematics  can  be  viewed  as  an  aid  to  decision  making.  By  using  mathematics 
the  decision  maker  can  obtain  a  very  special  type  of  information  which  comes 
from  applying  a  logically  consistent  interpretation  to  some  part  of  his  problem. 
Though  this  logical  consistency  is  considered  desirable  because  it  seems  to 
lead  to  a  certain  generality  and  workability  of  the  solutions,  this  is  no  place 
to  digress  into  a  discussion  of  the  possibly  subtle  reasons  why  this  is  so. 

It  will  be  useful,  however,  to  consider  in  some  detail  the  historical 
development  of  one  area  of  applied  mathematics,  statistics.  The  earliest 
stage  of  development  of  statistics  was  concerned  primarily  with  use  of  numerical 
indices  to  summarize  data.  Decisions  were  then  made  on  the  basis  of  these 
indices.  Statisticians  soon  began  using  mathematics  in  the  form  of  probability 
theory  to  provide  theoretical  models  for  the  sampling  and  data-generating 
processes  frequenty  encountered  in  practice.  These  models  proved  to  be 
particularly  useful  as  guides  in  the  design  of  experiments  and  in  the  selection 
of  indices.  Further,  knowledge  of  the  underlying  theoretical  model  resulted 
in  the  index  being  more  informative  to  the  decision  maker.  This  development 
of  theoretical  models  for  data-generating  processes  continues  to  this  day  and, 
of  course,  is  a  very  vital  and  useful  part  of  statistics. 

Another,  quite  important,  line  of  development  in  statistics,  that  of 
relating  the  statistical  procedure  more  and  more  closely  to  the  decision 
problem  of  the  user  of  statistics,  received  great  impetus  from  the  work  of 
Neyman  and  Pearson  and  somewhat  later  from  the  work  of  Wald.  The  trend 
continued  until  we  now  have  contemporary  decision  theory^  as  the  most  complete 
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and  unified  matheniatical  treatment  of  t!ie  decision  process  yet  achieved.  It 
is  a  significant  extension  of  the  earlier  statistical  theories  in  that  by 
explicitly  incorporating  the  decision  maker’s  judgment  and  experience  (in 
the  form  of  probabilities  and  utilities)  into  the  formulation  of  the  decision 
problem  it  is  able  to  deal  not  only  with  decisions  made  on  the  basis  of 
sampling  data  but  also  with  decisions  which  must  be  made  in  the  absence  of 
data  obtained  from  sampling  and,  of  course,  with  intermediate  decisions 
as  to  whether  or  not  to  obtain  additional  information  before  making  a  terminal 
decision.  In  this  sense,  therefore,  decision  theory  can  be  seen  to  provide 
the  applied  mathematics  of  the  decision  process.* 

* 

Some  traditionally  trained  statisticians  object  to  this  decision-theoretic 
extension  of  classical  statistical  theory  because,  in  their  view,  it  makes 
their  methods  of  analysis  seem  less  objective.  Thus,  there  is  controversy 
among  statisticians  as  to  whether  to  use  a  decision-theoretic  or  a  ’’classical'* 
procedure  in  a  particular  application.  Since  we  are  considering  the  use  of 
mathematics  to  aid  the  human  decision  process,  it  is  important  to  evaluate 
the  implications  of  this  controversy  about  statistical  practice. 

First,  it  should  be  appreciated  that  in  those  cases  in  which  a  decision 
must  be  made  without  the  assistance  of  data  obtained  by  sampling  there  is  no 
conflict.  Classical  statistical  theory  does  not  apply.  Second,  in  those  cases 
in  which  sampling  data  is  available  there  is  n^  logical  inconsistency  between 
procedures  recommended  by  decision  theory  and  by  classical  statistical  theory. 
The  procedures  can  be  viewed  as  differing  only  with  respect  to  the  type  of 
intuitive  judgment  required  of  the  decision  maker,  e.g.,  the  specification  of 
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probabilities  and  utilities  for  the  decision-theoretic  analysis  versus  tlie 

specification  of  error  rates  or  confidence  coefficients  for  the  I'eynan-Pearson 
o 

analysis.^  Thus,  if  v/c  agree  that  a  decision  niaker  should  be  allowed  to 
express  those  intuitive  judgments  that  he  finds  most  natural  and  to  deal 
with  his  problem  in  those  terms,  there  remains  no  basic  conflict  between 
the  statistical  theories. 


Discrepancy  between  Potential  and  Actual  Use  of  Decision  Theory 

Tims,  decision  theory  is  a  natural  development  of  the  growth  of  statistical 
theory  but  more  importantly  it  represents  nothing  more,  nor  less,  than  the 
extension  of  logic  and  mathematics  into  the  domain  of  choice  and  decision. 

It  deals  with  the  choice  of  the  proper  course  of  action  to  achieve  a  certain 
end  result.  The  scope  of  application  of  decision  theory  is  constantly 
expanding  through  the  definition  and  solution  of  new  classes  of  decision 
problems.  The  problems  that  we  routinely  approach  today  were  considered 
immensely  complex  just  a  few  years  ago.  New  formulations  and  accretions  to 
decision  theory  will  undoubtedly  aid  in  the  continued  expansion  of  the  domain 
of  application  of  this  area  of  applied  mathematics. 

In  summary,  the  conceptual  tools  of  decision  theory  are  undergoing 
continual  improvement  and  it  is  difficult  to  perceive  any  insurmountable 
obstacle  to  the  continuing  increase  in  the  scope  of  potential  application  of 
the  mathematics.  Notice  that  I  used  ‘’potential”  to  qualify  application. 

Here  I  refer  to  the  obvious,  but  sometimes  forgotten,  fact  that  the  mere 
existence  of  a  certain  tool  does  not  guarantee  its  use  in  practice. 

The  Annals  of  Mathematical  Statistics  represents  a  vast  pool  of  refined 
techniques  for  the  statistical  analysis  of  data.  To  one  with  any  appreciation 
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of  tlie  body  of  potential  techniques  represented  by  the  Annals ,  an  exanination 


of  the  statistical  techniques  actually  used  in  practice  by  engineers, 
scientists,  and  even  statisticians  nust  be  a  rather  discouraging  experience. 
Cacii  of  us  probably  has  his  own  favorite  estimate  of  the  lag  existing  between 
the  development  of  a  new  statistical  technique  and  its  general  utilization  by 
practitioners.  My  favorite  is  three  decades,  but  I  won’t  quibble.  One  decade 
or  ten  decades,  the  lag  seems  excessive. 

So,  in  contrast  to  a  rather  encouraging  view  of  the  future  development 
of  the  mathematical  theory  of  optimal  decision  making,  we  have  a  very 
discouraging  picture  of  the  extent  of  future  applications  of  this  theory. 

If  conditions  remain  as  in  the  past,  decision  theory  will  not  be  widely 
applied  in  this  generation  or  by  this  generation.  As  a  new  generation  of 
managers,  engineers,  and  scientists  is  trained  in  the  universities  to  use 
this  new  theory  we  will  see  more  application,  but  even  granted  this  training, 
the  use  of  decision  tlieory  will  be  restricted  to  the  more  complicated,  the 
more  important  problems.  And  this  is  as  it  should  be,  unless — . 

The  Cost  and  Gains  of  Applying  Decision  Theory 

The  concepts  and  methods  of  decision  theory  can,  of  course,  be  applied 
to  the  decision  of  whether  or  not  to  use  the  mathematics  of  decision  theory 
as  an  aid  in  making  a  particular  decision.  Or,  possibly,  the  choice  is 
between  a  simpler  or  a  more  complete  formulation  and  analysis  of  the  particular 
decision  problem.  In  either  case,  one  can  formulate  the  choice  in  terms  of 
two  marginal  costs ^  the  marginal  cost  of  foregoing  the  decision- theoretic 
analysis  and  the  marginal  cost  involved  in  carrying  out  the  decision- theoretic 
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analysis.  In  both  cases,  especially  the  first,  there  is  always  a  loss  involved 
in  foregoing  a  decision-theoretic  analysis.  Given  the  formulation  of  the 
particular  decision  problem,  the  decision-theoretic  analysis  must  recommend 
the  course  of  action  yielding  the  largest  possible  expected  payoff  whereas 
an  intuitive  or  a  less  complete  analysis  will  result  either  in  the  choice  of 
this  optimal  course  of  action  or  in  one  which  yields  a  smaller  expected  payoff. 
In  this  latter  case,  a  non-zero  cost  is  incurred  in  foregoing  the  decision- 
theoretic  analysis.  In  other  words,  one  cannot  do  better,  by  definition, 
than  to  choose  the  course  of  action  recommended  by  the  decision-theoretic 
analysis .* 

* 

Of  course  this  is  conditional  upon  one  accepting  the  particular 
formulation  and  values  of  the  decision  problem  under  analysis. 


Though  it  is  known  that  typically  there  will  be  a  loss  incurred  by 
foregoing  a  decision- theoretic  analysis,  the  actual  magnitude  of  this  loss 
is  unknown  unless  the  decision-theoretic  analysis  is  performed  and  the  results 
compared  with  that  yielded  by  an  intuitively  chosen  course  of  action.  Thus, 
a  reasonable  strategy  to  adopt  in  this  kind  of  situation  is  to  try  to  minimize 
the  other  cost,  the  marginal  cost  involved  in  carrying  out  the  decision-theoretic 
analysis.  Ideally,  this  cost  should  be  reduced  to  as  near  zero  as  possible. 

Then  the  best  choice  would  be  to  conduct  the  decis ion-theoretic  analysis  in 
all  cases  in  which  the  cost  of  application  were  zero  or  insignificant. 

In  considering  the  reduction  of  this  marginal  cost  of  application  it  is 
useful  to  analyze  the  components  that  make  up  the  total  cost  of  carrying  out 
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a  full-fledged  decision-theoretic  analysis  of  a  decision  problen.  It  will  be 
convenient  to  divide  this  cost  into  two  components;  personal  cost  and  non¬ 
personal  cost,  where  personal  cost  represents  tlie  marginal  cost  to  the 
decision  maker  while  non-personal  cost  represents  the  marginal  cost  to  the 
organization,  institution,  or  society. 

As  to  the  personal  marginal  cost  to  the  decision  maker,  consider  what 
he  must  be  able  to  do  in  order  to  carry  out  a  decision-theoretic  analysis. 

He  must  learn  the  concepts  and  techniques  of  decision  theory  and  related 
mathematics.  He  must  learn  how  to  utilize  the  devices  and  techniques  he  has 
available  for  carrying  out  the  decision-theoretic  analysis.  He  must  spend 
time  and  effort  formulating  the  particular  decision  problem,  assessing  the 
probabilities  and  utilities,  and  applying  the  various  computations  necessitated 
by  the  theory.  Obviously,  the  application  of  decision  theory  to  a  specific 
decision  probelm  is  typically  a  very  costly  process  for  the  decision  maker. 

The  non-personal  marginal  cost  to  the  organization,  institution,  or 
society  is  in  many  respects  related  to  the  personal  costs  described  above. 

For  example,  the  several  years  of  higher  education  that  may  be  required  for 
mastery  of  decision  theory  may  be  considered  a  cost  to  society.  The  time 
required  for  a  manager,  say,  to  apply  decision  theory  to  one  of  his  management 
problems  may  be  seen  as  a  cost  to  his  organization. 

Some  non-personal  costs  have  the  characteristic  that  they  can  be  used  to 
reduce  other  costs  of  application,  both  personal  and  non-personal.  For  example 
the  institution  may  incur  the  cost  of  employing  a  consulting  statistician  in 
order  to  reduce  the  training  requirements  on  other  employees.  The  installation 
of  a  computer  facility  along  with  a  programming  staff  serves  as  another  example 
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The  use  of  a  computer  can  result  in  fantastic  reduction  in  the  cost  of 
computation  though  the  costs  of  programming  have  tended  to  restrict  its 
use  to  very  extensive  and  complex  computations  in  scientific  applications 
or  to  highly  repetitive  routine  operations  in  business  applications.  In 
the  case  of  very  extensive  computations,  the  advantage  of  using  a  computer 
appears  not  in  a  reduction  in  the  cost  of  an  on-going  operation  but  in 
allowing  for  more  complex  and  thorough  analyses  which  would  never  have  been 
performed  (both  for  economic  and  other  reasons)  without  the  aid  of  the  computer. 
In  the  case  of  highly  repetitive,  routine  operations,  the  advantage  of  using 
a  computer  lies  in  its  low  cost  of  computation  plus  the  fact  that  the  intial 
programming  investment  can  be  amortized  over  many  repeated  applications. 

Reducing  Costs  of  Application  Through  the  Use  of  Computers 

Now  computers  have  certainly  been  used  to  perform  decision-theoretic 
computations.  However,  due  to  the  great  cost  of  programming,  these  applica¬ 
tions  have  been  restricted  to  the  analysis  of  rather  complex  and  important 
decision  problems.  Furthermore,  because  of  the  apparent  rarity  of  occurrence 
of  these  complex  decision  problems  there  seems  to  be  no  possibility  of 
amortizing  the  cost  of  programming  over  many  different  applications.  Thus, 
the  cost  of  a  decision-theoretic  analysis  remains  quite  high. 

Suppose  that  we  consider  increasing  the  investment  in  programming  in 
order  to  increase  the  scope  of  application  of  the  programmed  techniques.  It 
may  be  possible  to  so  increase  the  number  of  situations  in  which  the  techniques 
can  be  used  that  the  programming  and  hardware  costs  per  application  become  quite 
negligible. 
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Two  ingredients  seem  to  be  necessary  to  the  success  of  such  an  attempt. 
First,  the  programmed  techniques  need  to  be  of  such  generality  that  they  may 
be  sensibly  applied  in  a  wide  variety  of  situations.  Second,  the  programmed 
techniques  must  be  such  that  they  result  in  a  significant  reduction  in  the 
personal  cost  of  application. 

Some  of  the  potentialities  inherent  in  such  an  approach  are  illustrated 

by  STATPAC,  a  scope  directed  on-line  system  for  data  analysis,  designed  by 

3 

John  B.  Goodenough  of  the  Decision  Sciences  Laboratory.  The  original  data 
from,  say,  a  psychological  experiment  is  entered  into  the  computer  by  means 
of  punched  paper  tape.  The  psychologist  using  scope  and  light  pen  can  then 
perform  a  wide  variety  of  mathematical  operations  on  his  data.  Results  of 
interest  are  either  typed  or  punched  out  at  the  option  of  the  psychologist. 

One  significant  achievement  of  this  system  is  that  the  only  special 
knowledge  required  of  the  user  is  that  he  know  how  to  get  his  data  punched 
on  paper  tape  in  the  proper  format,  how  to  start  the  computer  and  read  in 
paper  tape,  and  how  to  use  the  light  pen.  In  particular,  the  user  does  not 
need  to  know  anything  at  all  about  computer  programming  nor  does  he  need 
to  learn  the  vocabulary  and  grammar  of  a  '*problem-oriented**  language.  The 
user  does  need  to  know  something  about  elementary  statistical  analysis  of 
data  though  even  this  requirement  seems  to  be  minimal  since  a  user,  given 
that  he  knows  enough  to  suspect  that  using  STATPAC  may  help  him,  may  ”learn 
by  doing”  in  using  the  system.*  However,  this  type  of  learning  is  clearly 
limited  to  the  more  elementary  operations.  The  operations  involved  and  the 
interpretation  and  meaning  of  a  derived  inferential  value  like  Student's  ^ 
cannot  be  learned  in  this  manner.  In  summary,  STATPAC  makes  the  power  of 
the  computer  available  for  data  analysis  under  the  control  of  a  non-programming 
scientist. 
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*  STATPAC  has  complete  program  protection  from  all  user  errors  along  with 
diagnostics  for  many  illegal  operations.  Thus,  a  user  may  attempt  to  perform 
any  operation  in  any  order  and,  if  successfully  completed,  examine  the  result 
on  the  scope  display. 


It  is  important  to  realize  that  STATPAC  is  not  suited  to  the  complex 
and  extensive  data  analyses  of  the  type  usually  programmed  for  computer  solution.* 
Its  domain  of  application  lies  among  those  computations  that  would  commonly 
be  performed  by  hand  or  with  the  aid  of  a  desk  calculator.  The  use  of  a 
computer  has  generally  not  been  competitive  with  manual  computation  unless 
the  problem  has  required  hundreds  or  thousands  of  hours  of  human  computations. 

Thus,  another  significant  achievement  of  STATPAC  is  that,  by  eliminating 
marginal  costs  of  programming  and  by  reducing  to  a  minimum  the  requirement 
for  special  knowledge  on  the  part  of  the  potential  user,  the  user  of  the 
computer  offers  advantages  even  in  the  case  of  data  analysis  problems 
requiring  only  a  few  hours  of  hand  computation. 

* 

It  appears  that  even  some  of  the  more  extensive  data  analysis  techniques 
might  benefit  from  programming  to  reduce  the  cost  of  application.  For  example, 
scope  directed  programs  for  analysis  of  variance  and  for  information- 
theoretic  analysis  of  data  are  being  developed  by  Charles  R,  Brown  at  the  Decision 
Sciences  Laboratory . 


The  net  effect  of  STATPAC  has  been  to  bring  in  a  new  group  of  users  of 
our  computer,  i.e.,  those  staff  members  who  have  not  mastered  computer  programming 
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and  operation.  Furtherr.iore ,  many  experiments  are  more  thoroughly  analyzed 
and  much  more  quickly  analyzed  than  ever  before. 

A  decis ion-theoretic  analysis  of  a  problem  has  much  in  common  with  the 
statistical  analysis  of  data.  They  both  employ  mathematical  procedures  of 
varying  degrees  of  sophistication.  Both  are  potentially  applicable  to  a 
wide  variety  of  situations.  Thus,  considering  the  success  of  STATPAC  it 
appears  reasonable  to  apply  similar  principles  to  the  development  of  a 
system  for  the  on-line  analysis  of  decision  problems. 

CORTEX,  Computer-based  Optimization  Routines  and  Techniques  for 
Effective  X  (=  command,  management,  engineering  design,  research,...),  is 
just  such  a  system  under  development  and  study  at  the  Decision  Sciences 
Laboratory . 

CORTEX  in  Operation 

l^at  follows  is  a  brief  description  of  how  CORTEX  as  presently  conceived 
operates.  The  description  is  brief  both  because  it  is  best  to  avoid  getting 
lost  in  the  details  and  because  many  details  are  subject  to  change  as  we 
attempt  to  make  CORTEX  easier  to  use.  However,  some  of  the  operations, 
particularly  the  assessment  of  probabilities,  are  discussed  in  some  detail  in 
later  sections. 

A  potential  user,  hereafter  called  the  decision  maker,  sits  down  before 
a  remote  computer-driven  scope  and  light  pen  station.  All  communication 
between  the  computer  and  the  decision  maker  occurs  through  the  light  pen  and 
scope  except  for  certain  final  results  which  are  needed  in  hard-copy  form. 

The  decision  maker  requests  CORTEX  by  throwing  a  switch  on  the  console.  If 
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the  computer  is  not  busy,  CORTEX  is  now  available  and  the  scope  and  light 
pen  are  active, 

I,  Formulation  of  the  basic  decision  problem, 

A,  CORTEX  asks  the  decision  maker  to  specify  the  courses  of  action 
he  wishes  to  consider.  If  there  is  a  small  number  of  courses  of 
action,  the  decision  maker  may  identify  them  by  "typing"  them 

on  the  face  of  the  scope  with  the  light  pen.  If  there  is  a 
large  number  of  potential  courses  of  action,  e,g,,  when  the  problem 
is  to  choose  a  certain  amount  to  purchase,  stock,  employ,  or 
deploy,  the  decision  maker  may  identify  them  by  specifying  the 
range  and  step  size.  It  should  be  noted  that  the  selection  of 
these  courses  of  action  is  tentative  since  the  decision  maker 
at  a  later  stage  may  delete  some  and/or  add  others  and  reformulate 
the  decision  problem  where  necessary, 

B,  CORTEX  asks  the  decision  maker  to  identify  those  states  of  the 
world  which  determine  the  desirability  or  undersirability  of  the 
courses  of  action,  CORTEX  queries  the  decision  maker  to  insure 
that  the  states  are  logically  mutually  exclusive  and  exhaustive. 

The  identification  of  the  states  of  the  world  is  again  tentative 
and  may  be  changed  by  the  decision  maker  at  a  later  stage. 

C,  CORTEX  provides  two  classes  of  methods  to  aid  the  decision  maker 
in  assessing  the  probabilities  of  the  states  of  the  world.  If  the 
decision  maker  chooses  to  use  both  methods  the  results  are  checked 
for  consistency  and  if  the  decision  maker  is  concerned  at  the  lack 
of  agreement  between  the  two  sets  of  probabilities  he  may  choose 
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either  to  resolve  the  inconsistency  in  some  manner,  to  retain 
the  two  sets  for  a  sensitivity  analysis  at  a  later  stage,  or 
to  repartition  the  states  of  the  world  so  that  he  may  more 
easily  assess  the  probabilities. 

D.  The  consequences  of  each  possible  outcome,  i.e.,  the  joint 
occurrence  of  a  course  of  action  and  a  state  of  the  world, 
are  assessed  numerically  by  the  decision  maker.  CORTEX  provides 
tests  to  determine  whether  monetary  costs  and  gains  are  adequate 
or  whether  more  general  utilities  are  needed.*  CORTEX  tests 
automatically  for  dominated  courses  of  action,  i.e.,  a  course 
of  action  which  yields  no  more  desirable  consequences  than 
another  course  of  action  for  each  of  the  possible  states  and 
yields  a  less  desirable  consequence  for  at  least  one  state,  and 
calls  these  to  the  attention  of  the  decision  maker  for  possible 
elimination  from  consideration  or  for  re-evaluation.  The  decision 
maker  may  find  that  he  has  partitioned  the  states  in  a  manner  that 
makes  it  difficult  for  him  to  specify  the  consequences.  In  this 
case  he  may  choose  to  collapse  some  of  the  states,  in  which  case 
the  new  probability  is  automatically  computed  by  CORTEX,  or  to 
partition  further  one  or  more  states,  in  which  case  the  additional 
probabilities  are  requested  by  CORTEX. 


*  In  those  applications  where  the  consequences  can  be  expressed 
in  monetary  terms,  simple  reference  contracts  are  used  to  determine 
whether  or  not  the  decision  maker’s  utilities  are  linear  with 
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respect  to  monetary  gains  and  losses  over  the  range  necessitated 

4 

by  the  particular  decision  problem.  If  approximate  linearity 

is  found  then  the  monetary  values  can  serve  as  the  numerical 

values  of  the  consequences.  If,  however,  significant  departures 

from  linearity  are  indicated  then  an  approximate  utility  function 

must  be  determined  for  the  decision  maker  or  the  decision  problem 

must  be  reformulated  since  such  deviations  may  be  due  to  the 

A 

partial  formulation  of  interdependent  decision  problems  or  to  the 
effect  of  induced  utilities^. 


E,  Once  the  decision  maker  is  satisfied  with  the  problem  formulation 
developed  above  he  can  request  CORTEX  to  compute  and  to  list  (or, 
in  some  problems,  to  graph)  the  expected  utilities  for  each  course 
of  actibn.  The  optimal  course  of  action  yielding  the  largest 
expected  utility  is  emphasized  by  additional  intensification  on 
the  scope.  In  addition,  the  expected  loss  or  cost  of  uncertainty 
for  the  optimal  course  of  action  is  displayed.  At  the  option  of 
the  decision  maker,  these  results  may  be  obtained  in  a  hard-copy 
form  for  further  reference. 

F.  If  the  decision  maker  feels  any  uncertainty  about  the  decision 
problem  or  if  he  would  like  to  understand  better  the  nature  of 
the  decision  problem,  he  may  perform  a  sensitivity  analysis  by 
going  back  to  earlier  stages  of  CORTEX,  changing  the  courses  of 
action,  the  states  of  the  world,  the  numerical  values  of  the 
consequences,  or  the  probabilities  of  the  states,  and  then  observing 
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the  resulting  changes  in  the  expected  utilities  and  the  optimal 
course  of  action*  If  the  decision  maker  judges  that  the  effect 
of  these  changes  is  slight,  he  need  feel  no  concern  about  his 
uncertainty.  If,  on  the  other  hand,  the  decision  maker  finds 
that  the  changes  produce  major  effects  in  the  solution  of  his 
decision  problem,  he  might  consider  ways  of  obtaining  additional 
information  in  order  to  reduce  his  uncertainty. 

II.  Evaluation  of  Potential  Information  about  the  States. 

A.  Uncertainty  about  the  states  of  nature  can,  of  course,  be  reduced 
by  gathering  additional  information  about  the  states,  possibly 
but  not  necessarily  through  sampling.  Therefore,  at  this  stage 
CORTEX  displays  again  the  cost  of  uncertainty,  reminds  the  decision 
maker  that  this  value  represents  the  largest  possible  gain  that 
could  result  from  obtaining  additional  information  about  the  states, 
and  then  queries  the  decision  maker  about  his  desire  to  consider 
the  value  of  additional  information.  If  the  cost  of  uncertainty 
appears  large  to  the  decision  maker  relative  to  what  he  thinks  that 
additional  information  might  cost,  the  decision  maker  may  choose 

to  consider  the  possibility  of  obtaining  additional  information 
on  the  states  of  the  world. 

B.  CORTEX  then  queries  the  decision  maker  about  the  nature  of  the 
additional  information  that  may  be  obtained.  A  major  distinction 

is  made  between  sampling  experiments  involving  repeated,  independent 
observations  and  other  non-sampling  types  of  information.  In  the 
case  of  sampling,  the  decision  maker  may  assess  the  conditional 
probabilities  of  the  data  given  each  state  of  the  world  by  using 
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aids  similar  to  those  described  in  the  section  on  the  assessment 


of  probabilities  or  he  may  choose  to  specify  the  type  of  data- 
generating  process  and  the  parameters  of  the  process.*  In  the 
case  of  non-sampling  information,  there  are  no  data-generating 
processes  available  so  the  decision  maker  must  assess  at  least 
some  of  the  conditional  probabilities. 


*  CORTEX  contains  the  standard  data-generating  processes  such 
as  the  binomial,  multinomial,  Pascal,  Poisson,  normal,  and 
multivariate  normal. 

Once  the  conditional  probabilities  have  been  specified,  the 
decision  maker  may  request  a  consistency  check  based  on  hypothetical 
decisions  in  a  simplified  experiment  as  described  in  the  section  on 
the  assessment  of  probabilities. 

C.  CORTEX  interrogates  the  decision  maker  as  to  the  estimated  cost 
of  the  potential  Information.  In  the  case  of  sampling,  the 
decision  maker  specifies  the  marginal,  or  if  he  prefers  the 
cumulative,  cost  of  sampling  and  the  maximum  feasible  sample 
size.  The  cost  of  sampling  may  be  a  fixed  amount  per  observation, 
a  fixed  amount  per  group  of  observations,  or  in  general  any 
function  of  the  cumulative  number  of  observations.  CORTEX 
provides  a  graphical  response  mode  for  the  more  complex  functions 
so  that  the  decision  maker  can  draw  the  function  on  the  face  of 
the  computer-driven  scope. 
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D.  Now  that  all  of  the  necessary  information  is  available,  CORTEX 
computes  and  displays  all  of  the  relevant  expected  utilities. 

In  the  case  of  sampling,  for  example,  CORTEX  displays  a  graph 
showing  the  expected  utility  of  the  optimal  strategy  for  each 
fixed  sample  size  from  one  up  to  and  including  the  largest 
feasible  sample  size.  The  largest  expected  utility,  correspond¬ 
ing  to  the  optimal  sample  size  for  fixed  sampling,  is 
emphasized  by  additional  intensification  and  the  location  of 
a  pointer.  The  expected  utility  associated  with  optimal  sequential 
sampling  is  displayed  as  a  horizontal  dashed  line  which  may  or 
may  not  intersect  the  expected  utility  function  for  fixed 
sampling.  If  it  does  then  fixed  sampling  is  to  be  preferred; 
otherwise  sequential  sampling  is  better.*  In  addition  to  these 
expected  utilities  for  sampling  experiments,  CORTEX  displays 
along  the  vertical  axis  the  expected  utilities  for  the  best  and 
worst  courses  of  action  under  the  condition  of  no  sampling 
information.  By  this  means  the  decision  maker  is  provided  with 
information  sufficient  to  determine  whether  or  not  it  is  worthwhile 
to  obtain  additional  information  about  the  states  of  the  world 
through  use  of  a  sampling  experiment.  Additionally,  the  decision 
maker  is  provided  with  information  about  the  expected  performance 
of  fixed  and  sequential  sampling  procedures  as  a  basis  for  choosing 
the  best  experiment. 

*  Sequential  sampling  is  no  worse  and  frequently  better  than 
fixed  sampling  when  cost  of  sampling  is  proportional  to  sample 
size.  However,  in  the  case  of  more  complicated  cost  structures, 
fixed  sampling  may  be  superior  to  sequential  sampling. 
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E.  If  the  decision  maker  feels  any  uncertainty  about  his 
specification  of  the  conditional  probabilities  of  the  data  given 
the  states  of  the  world  or  of  the  marginal  costs  of  sampling 
and  the  maximum  sample  size,  he  may  as  before  request  a  hard¬ 
copy  summary  of  the  results  of  the  current  analysis  and  then 
return  to  an  earlier  stage  of  CORTEX  to  perform  a  sensitivity 
analysis.  In  an  analogous  manner,  the  decision  maker  may  return 
to  an  earlier  stage  and  insert  new  probabilities  and  costs  of 
sampling  in  order  to  evaluate  an  alternative  experimental  design 

in  the  expectation  of  finding  a  procedure  yielding  better  performance. 

F.  Once  the  decision  maker  is  satisfied  with  the  analysis  of  the 
decision  problem  he  may  request  a  hard-copy  record  of  the  results 
of  the  analysis  including  a  complete  statement  of  the  optimal 
decision  rules  and  expected  utilities  conditional  upon  the  observed 
data.  If  sequential  sampling  is  indicated,  the  decision  maker 

may  request  CORTEX  to  compute,  display,  and  record  the  probability 
distribution  of  optimal  sample  sizes  and  the  expected  optimal 
sample  size.  This  information  can  be  used  to  guide  the  conduct 
of  the  sampling  experiment  that  is  under  consideration. 
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The  Assessment  of  Probabilities 


The  assessment  of  probabilities  is  certainly  one  of  the  more  critical 
and  less  developed  aspects  of  decision  making.  It  is  through  probabilities 
that  much  of  the  decision  maker’s  judgement  and  experience  is  given  numerical 
express  and  brought  to  bear  upon  the  mathematical  solution  of  his  decision 

A 

problem.  Thus,  it  may  prove  worthwhile  to  consider  in  some  detail  a  number 
of  the  methods  that  are  available  for  assisting  the  decision  maker  in 
expressing  his  probabilities. 

First,  take  the  case  of  a  discrete  probability  distribution.  Let  there 
be  a  set  of  mutually  exclusive  and  collectively  exhaustive  events  each  of 
which  corresponds  to  a  different  state  of  the  world,  say,  or  to  one  of  the 
possible  outcomes  of  a  sampling  study.  At  the  given  moment  of  assessment, 
the  decision  maker  may  not  know  the  actual  state  of  the  world  but,  since 
the  set  of  events  is  mutually  exclusive  and  collectively  exhaustive,  one 
and  only  one  state  of  the  world  must  exist.  In  order  to  utilize  CORTEX, 
or  in  fact  any  method  based  on  the  theory  of  probability,  the  decision 
maker  must  assign  a  probability  value  to  each  event. 

One  method  used  in  CORTEX  is  based  on  having  the  decision  maker  express 
his  preference  between  two  simple  hypothetical  experiments,  one  having 
outcomes  dependent  upon  the  original  set  of  events,  the  other  having  outcomes 
dependent  upon  a  reference  set  of  events.  Rather  than  have  the  decision 
maker  deal  simultaneously  with  all  of  the  events  in  the  original  set  we  have 
the  decision  maker  deal  with  the  events  two  at  a  time  by  contrasting  the 
occurrence  of  a  particular  event  with  the  non-occurrence  of  the  particular 
event.  Thus,  CORTEX  begins  by  describing  two  hypothetical  experiments  for 
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the  decision  maker’s  consideration.  The  first  experiment  yields  $100  if 
a  specified  original  event  occurs  and  $0  if  it  does  not  occur.  The 
second  experiment  yields  $100  if  a  specified  reference  event  occurs  and  $0 
if  it  does  not  occur. 

The  reference  event  used  by  CORTEX  is  based  upon  a  well-understood 
data-generating  process  and  is  explained  to  the  decision  maker.  A  line 
is  displayed  on  the  scope.  The  line  is  divided  into  left-hand  and  right- 
hand  segments  by  a  movable  index.  One  of  the  points  on  the  line  will  be 
chosen  at  random,  i.e.,  each  point  is  as  likely  to  be  selected  as  any 
other  point.  The  decision  maker  may  observe  this  process  operating  at 
the  rate  of  several  selections  per  second  for  any  length  of  time  he 
desires.  The  reference  event  occurs  when  the  random  point  falls  to  the 
left  of  the  movable  index.  Thus,  the  second  experiment  yields  $100  if 
the  random  point  is  located  on  the  line  segment  to  the  left  of  the  index 
and  $0  if  located  on  the  line  segment  to  the  right  of  the  index.  It 
should  be  noted  that  the  probability  of  the  reference  event  is  proportional 
to  the  length  of  the  line  segment  extending  from  the  left  end  of  the  line 
over  to  the  index. 

Now,  it  should  be  clear  that  the  expected  value  to  the  decision  maker 
of  the  first  experiment  is  proportional  to  the  probability  of  the  specified 
original  event  while  the  expected  value  of  the  second  experiment  is  proportional 
to  the  probability  of  the  specified  reference  event.  The  decision  maker 
prefers,  of  course,  the  experiment  possessing  the  largest  expected  value. 

Since  the  expected  value  of  the  second  experiment  depends  upon  the  location 
of  the  index,  there  will  exist  at  least  one  setting  of  the  index  at  which 
the  decision  maker  will  not  prefer  the  first  experiment  and  there  will 
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exist  at  least  one  setting  of  the  index  at  which  he  will  not  prefer  the 
second  experiment.*  More  importantly,  there  will  exist  a  setting  of 
the  index  or  a  range  of  settings  at  which  the  decision  maker  will  be 
indifferent  between  the  two  experiments.  Oace  this  index  setting  represent¬ 
ing  practical  indifference  has  been  determined  the  ratio  of  the  length  of 
the  line  segment  to  the  left  of  the  index  setting  to  the  length  of  the 
total  line  is  taken  to  be  the  probability  of  the  original  event. 


*  This  sentence  is  worded  in  this  somewhat  awkward  manner  in  order  to 
include  the  possibility  that  the  probability  of  the  original  event  is 
zero  or  one. 


The  decision  maker  can  use  one  or  both  of  two  modes  of  operation  of 
CORTEX  to  arrive  at  this  index  setting.  In  the  first  mode  of  operation 
the  probability  of  the  origninal  event  is  determined  by  a  method  of 
successive  approximation.  CORTEX  sets  the  index  at  the  middle  of  the  line 
corresponding  to  a  probability  of  the  reference  event  equal  to  one-half 
and  queries  the  decision  maker  as  to  his  preference  between  the  two 
experiments.  If  the  decision  maker  prefers  the  first  experiment  (indicating 
that  the  probability  of  the  original  event  is  greater  than  one-half) , 

CORTEX  sets  the  index  at  the  three-fourths  position.  If,  on  the  contrary, 
the  decision  maker  prefers  the  second  experiment  (indicating  that  the 
probability  of  the  original  event  is  less  than  one-half) ,  CORTEX  sets  the 
index  at  the  one-fourth  position.  Now,  the  decision  maker  indicates  his 
preference  between  the  first  experiment  and  the  new  second  experiment 
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based  on  the  adjusted  index  setting.  CORTEX  continues  this  search 
procedure  until  the  decision  maker  judges  that  his  preferences  are 
becoming  sufficiently  difficult  to  express  and/or  that  the  probability 
of  the  original  event  has  been  determined  with  sufficient  accuracy  and 
indicates  his  wish  to  discontinue  the  assessment. 

In  the  second  mode  of  operation  the  decision  maker  uses  his  light 
pen  to  move  the  index  to  any  desired  position  along  the  line.  The  decision 
maker  can  utilize  this  mode  of  operation  at  any  time,  either  before  or 
after  starting  in  the  successive  approximation  mode.  Once  the  decision 
maker  feels  that  he  has  achieved  a  satisfactory  setting  of  the  index,  he 
indicates  his  wish  to  discontinue  the  assessment.  Whereas  the  first  mode 
of  operation  is  particularly  useful  for  giving  a  decision  maker  unfamiliar 
with  the  assessment  of  probabilities  a  clear  understanding  of  the  nature 
of  the  process,  the  second  mode  of  operation  offers  certain  advantages  in 
terms  of  speed  and  ease  of  operation  to  a  decision  maker  experienced  in 
the  operation  of  this  portion  of  CORTEX. 

The  decision  maker  and  CORTEX  then  continue  on  in  this  manner  to 
obtain  a  probability  for  each  one  of  the  original  events.  Immediately 
upon  the  completion  of  the  last  assessment,  CORTEX  checks  to  determine 
whether  or  not  the  probabilities  sum  to  one,  informs  the  decision  maker  of 
this  condition,  and  if  required  offers  a  number  of  ways  for  the  decision 
maker  to  adjust  the  probabilities  so  that  they  do  in  fact  sum  to  one. 

This  first  method  of  assessing  a  discrete  probability  distribution  has 
many  advantages  in  its  favor,  e.g.,  the  choices  required  of  the  decision 
maker  are  of  a  particularly  simple  nature  and  are  independent  of  the  utility 
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values  that  the  decision  maker  may  attach  to  the  amounts  of  money  involved. 
However,  the  introduction  of  a  reference  set  of  events,  external  to  the 
decision  problem  at  hand,  may  sometimes  lead  to  inconsistent  and  biased 
measurements.  An  extensive  discussion  of  this  problem  may  be  found  in  the 
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articles  dealing  with  what  I  have  called-^  the  Chipman-Ellsberg-Fellner 
paradox.  In  essence,  the  two  sets  of  experiments  may  under  certain  condi¬ 
tions  prove  to  be  noncomparable  in  such  a  way  that  the  decision  maker 
will  encounter  great  difficulty  in  getting  the  probabilities  to  sum  to 
one.  In  this  case,  the  decision  maker  might  choose  either  to  reformulate 
his  decision  problem  or  else  to  utilize  the  assessment  procedure  described 
below. 

CORTEX,  at  present,  makes  available  one  other  method  for  the  assessment 
of  discrete  probability  distributions.  The  list  of  events  originally 
generated  by  the  decision  maker  is  displayed  on  the  scope.  Adjacent  to  the 
label  of  each  event  CORTEX  displays  a  line  segment,  each  initially  set  to 
be  of  the  same  length.  The  decision  maker  uses  the  light  pen  to  adjust 
the  lengths  of  these  lines  but  CORTEX  constrains  the  total  length  of  the 
lines  so  that  it  always  equals  a  fixed  value.  Adjacent  to  the  label  of 
each  event  CORTEX  also  displays  a  hypothetical  payoff  value  which  changes 
value  automatically  as  the  corresponding  line  length  is  changed  by  the 
decision  maker.  Each  payoff  value  represents  the  hypothetical  amount  of 
money  that  the  decision  maker  would  earn  if  the  corresponding  event  did 
in  fact  occur.  Thus,  the  decision  maker  adjusts  the  lengths  of  the  lines 
until  he  is  satisfied  that  the  corresponding  set  of  payoff  values  is  to 
his  best  advantage. 
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Now  the  critical  thing  to  realize  is  that  the  payoff  values  have 
a  very  special  relation  to  the  lengths  of  the  lines  such  that  the  decision 
maker  obtains  his  maximum  expected  payoff  if  and  only  if  the  length  of  the 
line  corresponding  to  each  and  every  event  is  proportional  to  the  probability 
of  that  event.*  In  other  words,  by  accurately  adjusting  the  lines  so  as 
to  honestly  reflect  his  probabilities,  the  decision  maker  maximizes  the 
expected  payoff  from  the  hypothetical  experiment.  It  should  be  mentioned 
that  the  payoffs  do  not  have  to  remain  hypothetical.  They  could,  for 
example,  represent  an  actual  bonus  or  fee  to  be  paid  to  the  decision  maker 
by  his  employer.** 

*  Tills  characteristic  of  the  payoff  function  has  been  termed  the 

9 

matching  property  ty  Toda.  Only  three  different  payoff  functions 

9,10 

have  been  found  which  have  this  property.  They  are  the  spherical  gain  , 

9  11  9  12 

the  logarithmic  loss  *  ,  and  the  quadratic  loss  * 

**  Grayson  in  his  study  of  the  application  of  decision  theory  to 

13 

drilling  decisions  by  oil  and  gas  operators  suggests  the  potential 
usefulness  of  similar  bonus  schemes  to  obtain  unbiased  probability 
estimates  from  consulting  geologists.  He  finally  concludes  however 
that  **...the  only  way  a  geologist  (or  any  decision  maker)  will  ever  give 
estimates  of  probabilities  that  approximates  his  true  beliefs  will  be 
through  understanding  and  acceptance  of  the  basic  idea.”  I  tend  to  agree. 


It  should  be  apparent  that  this  second  method  for  the  assessment  of 
discrete  probability  distributions  differs  from  the  first  method  in  a 
number  of  important  respects.  A  possibly  major  difference  is  that  the 
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second  method  is  self-contained  in  the  sense  that  it  does  not  require 
the  introduction  of  an  external  set  of  reference  events  as  does  the  first 
method.  The  accuracy  of  the  probabilities  yielded  by  this  second  method 
does  depend  upon  the  values  or  utilities  that  the  decision  maker  places  on 
the  amounts  of  money  involved  in  the  hypothetical  payoffs  whereas  in 
the  first  method  it  does  not.  Another  operational  difference  is  that 
in  the  second  method  the  decision  maker  deals  concurrently  with  all  of 
the  probabilities  under  the  automatic  constraint  that  they  sum  to  one. 

I  am  not  pointing  out  these  differences  to  bemoan  the  lack  of  a 
perfect  method.  On  the  contrary,  I  expect  that  some  decision  makers 
will  find  one  method  more  appropriate  and  natural  in  some  situations  while 
other  decision  makers  will  prefer  the  other  method.  Additionally,  a 
decision  maker  might  choose  to  use  both  of  the  methods  to  assess  the 
probabilities.  If  there  is  considerable  disagreement  between  the  two 
sets,  the  decision  maker  may  have  cause  for  concern  and  may  consider 
reformulating  his  decision  problem  or  he  may  choose  to  use  both  sets  of 
probabilities  in  a  sensitivity  analysis.  Finally,  experience  with  both 
methods  of  assessment  should  contribute  to  the  decision  maket’s  understanding 
of  the  nature  of  probability  and  uncertainty. 
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Consider  now  the  assessment  of  a  continuous  probability  distribution. 

Tliis  case  arises  when  the  events  under  consideration  are  quite  numerous, 
these  events  represent  different  amounts  or  magnitudes,  and  it  is  mathema¬ 
tically  convenient  to  deal  with  continuous  variables.  As  before,  CORTEX 
offers  the  decision  maker  two  methods  of  assessment. 

One  of  these  methods  is  quite  similar  in  spirit  to  the  first  method 
described  for  the  assessment  of  discrete  probability  distributions.  In 
fact,  it  employs  the  same  reference  experiment  which  is  represented  as 
before  by  a  line  divided  into  two  segments  by  an  index.  If  a  point  to  the 
left  of  the  index  is  selected  the  decision  maker  earns  $100  while  if  a 
point  to  the  right  of  the  index  is  selected  he  earns  $0.  CORTEX  initially 
sets  the  index  at  the  position  corresponding  to  a  probability  of  one-fourth. 
However,  in  this  method  of  assessment  the  decision  maker  has  no  control  over 
the  position  of  this  index. 

CORTEX  displays  a  second  line  above  the  line  representing  the  reference 
experiment.  This  new  line  represents  the  range  of  values  that  the  continuous 
events  may  assume  and  is  so  labeled.  There  is  also  an  index  on  this  line 
which  thus  represents  a  hypothetical  experiment  based  upon  the  probability 
distribution  under  assessment.  If  one  of  the  events  located  to  the  left  of 
this  index  actually  occurs  the  decision  maker  earns  $100  while  if  one  of  the 
events  located  to  the  right  of  this  index  occurs  he  earns  $0.  Therefore, 
the  expected  value  of  this  experiment  to  the  decision  maker  depends  upon  the 
probability  of  obtaining,  in  the  actual  decision  problem,  an  amount  or 
magnitude  less  than  that  specified  by  the  index  setting. 
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This  probability  is  assessed,  as  before,  by  determining  the  index 
setting  at  which  the  decision  maker  is  practically  indifferent  between 
the  two  hypothetical  experiments.  In  this  case,  however,  the  index  is 
varied  not  in  the  reference  experiment  but  in  the  experiment  based  on 
the  events  in  the  decision  problem  under  consideration.  Again,  CORTEX 
has  two  modes  of  operation  available  for  this  determination,  a  successive 
approximation  mode  and  a  continuous  adjustment  mode.  Either  may  be  used 
at  will  by  the  decision  maker. 

Once  the  decision  maker  has  achieved  a  satisfactory  setting  of  the  index, 
he  expresses  his  wish  to  discontinue  the  procedure  and  the  value  of  the  final 
setting  is  interpreted  by  CORTEX  as  the  first  quartile  of  the  continuous 
probability  distribution.  CORTEX  then  places  the  index  for  the  reference 
experiment  at  the  position  corresponding  to  a  probability  of  three-fourths  and 
the  procedure  is  repeated  once  more  to  determine  the  third  quartile  of  the 
continuous  probability  distribution. 

The  decision  maker  may  use  CORTEX  to  determine  the  second  quartile  or 
median  and,  in  fact,  any  fractile  of  the  distribution.  However,  if  the 
decision  problem  is  such  that  the  decision  maker  judges  that  the  continuous 
distribution  may  be  approximated  by  one  of  a  family  of  distributions  such  as 
the  beta,  gamma,  and  normal  then  the  two  quartiles  are  sufficient  information 
for  CORTEX  to  determine  the  complete  distribution  which  is  then  displayed 
along  with  the  corresponding  values  of  the  parameters  commonly  used  to  describe 
the  distribution.*  If  the  decision  maker  is  dissatisfied  with  the  appearance 
or  with  some  characteristic  of  this  distribution,  he  may  go  back  and  read  just 
one  or  both  of  the  quartiles  and  observe  the  resulting  changes  in  the  shape  of 
the  distribution  until  he  is  satisfied. 
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*  If,  for  example,  a  beta  distribution  were  being  employed,  CORTEX  would 
display  the  two  basic  parameters  2,  alternative  parameterization 

in  terras  of  2  and,  in  addition,  the  mean,  median,  mode,  and  standard 

deviation  of  the  beta  distribution  implied  by  the  two  quartiles  provided 
by  the  decision  maker. 


It  should  be  apparent  that  this  method  has  many  of  the  advantages  and 
disadvantages  that  characterize  the  corresponding  method  for  assessing 
discrete  probability  distributions  described  above.  For  example,  while 
the  choices  required  of  the  decision  maker  are  rather  simple,  the  introduction 
of  an  external  reference  experiment  may  produce  complications.  Thus,  CORTEX 
provides  a  second  method  for  assessing  continuous  probability  distributions 
which,  interestingly,  has  many  of  the  properties  of  the  alternative  method 
provided  in  the  case  of  discrete  probabilities. 

CORTEX  displays  a  line  representing  the  range  of  values  for  the  continuous 
events  in  the  decision  problem  under  consideration.  The  line  is  segmented 
by  two  indices  that  may  be  adjusted  by  the  decision  maker.  The  hypothetical 
experiment  is  such  that  the  decision  maker  earns  a  certain  constant  amount 
of  money  if  the  actual  event  assumes  a  value  lying  betx^;een  the  two  indices 
and  he  earns  nothing  if  the  actual  event  falls  outside  of  this  interval. 

In  any  case,  however,  the  decision  maker  must  pay  an  amount  of  money  which 
is  proportional  to  the  length  of  this  interval  bounded  by  the  two  indices. 

The  decision  maker  may  adjust  the  locations  of  the  indices  in  order  to 
maximize  the  expected  value  of  the  hypothetical  experiment.  Though  CORTEX 
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explains  and  demonstrates  the  optimal  strategy  to  the  decision  maker,  I 
will  not  attempt  to  describe  it  here.*  The  essential  thing  is  that 
the  decision  maker’s  settings  can  be  used  to  solve  for  the  complete 
distribution  which,  as  before,  is  displayed  along  with  a  set  of  relevant 
parameters.  Again,  the  decision  maker  nay  readjust  until  he  is  satisfied 
with  the  continuous  probability  distribution. 


This  method  has  been  derived  and  described  in  some  detail  by  Toda  . 


The  Future  Development  and  Evaluation  of  CORTEX 

As  implied  above,  the  total  value  of  the  application  of  a  system  such 
as  CORTEX  to  the  operations  of  an  organization  can  be  thought  of  in  terms 
of  the  total  marginal  gains  minus  the  total  marginal  costs  of  utilizing 
the  system.  Thus  the  decision  whether  or  not  to  have  such  a  system  depends 
upon  whether  this  difference  is  positive  or  negative.  Now,  these  marginal 
gains  of  application  are  seldom,  if  ever,  measured  in  an  unequivocal  manner 
and,  in  addition,  are  quite  expensive  to  measure.  Since  these  marginal 
gains  are  very  difficult  to  measure  in  application  to  particular  decision 
problems,  the  total  marginal  gain  resulting  from  the  application  to  many 
individual  decision  problems  is  practicably  unmeasurable  and  must  remain 
uncertain.  Thus,  the  total  effectiveness  of  CORTEX  is  as  uncertain  as  is 
the  total  effectiveness  of,  say,  linear  programming,  matrix  algebra,  or 
statistics . 

Of  course,  most  of  us  feel  that  mathematics  can  be  a  useful  tool  if 
intelligently  used.  This  judgement  gives  us  a  basis  for  assuming  that 
some  gains  are  non-zero  under  certain  conditions.  Therefore,  we  might 
attempt  to  increase  the  total  effectiveness  of  mathematics  both  by  increasing 
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the  nuTnber  of  intelligent  applications  of  the  T^iathematics  anc’  by  reducing 
the  cost  of  application  of  the  nathcniatics .  CORTEX  represents  an  attenpt 
to  do  just  this  for  the  mathenatics  of  decision  ipaking. 

By  any  criterion,  CORTEX  cannot  be  successful  if  it  is  not  used  to 
solve  decision  problems.  It  must  be  called  upon  quite  frequently  to  solve 
a  large  number  of  decision  problems.  \JIiether  or  not  it  will  be  used  to  solve 
a  particular  problem  will  be  decided,  in  effect,  by  the  answers  to  each  of  tv;o 
questions.  Are  the  techniques  provided  by  CORTEX  relevent  to  my  decision 
problem?  Is  CORTEX  easy  enough  to  use  in  x^/orking  Xv^ith  this  decision  problem? 
The  frequency  of  positive  replies  to  the  first  question  depends  in  large 
part  upon  the  generality  and  variety  of  the  algorithms  incorporated  into 
CORTEX*  wliile  the  frequency  of  positive  replies  to  the  second  question 
depends  in  large  part  upon  the  effectiveness  of  CORTEX  in  reducing  the 
personal  costs  of  application  referred  to  previously.  Tlius  the  development 
of  CORTEX  should  evolve  in  the  direction  of  including  more  and  more  algorithms 
and  in  the  direction  of  requiring  less  and  less  special  knowledge  on  the  part 
of  the  decision  maker. 

* 

It  also  depends  to  an  unknown  but  possibly  great  extent  upon  whether  or 

not  the  decision  maker  has  learned  to  viex^z  the  world  through  the  eyes  of 

14 

decision  theory.  Lest  an  opponent  of  decision  theory  rejoice  too  quickly, 

I  hasten  to  add  that  though  mathematics  may  be  afflicted  with  tunnel  vision 
it  yields  images  with  a  very  sharp  focus. 


The  attempt  to  reduce  the  amount  of  special  knowledge  required  of  the 
decision  maker  seems  certain  to  lead  sooner  or  later  to  a  system  that  has 
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so  many  explanations,  queries,  pronpts,  etc.  that  a  decision  maker 
experienced  in  the  use  of  the  system  will  find  it  quite  annoying  to 
use.  At  this  point,  preferably  earlier,  CORTEX  should  be  modified  so  that 
it  has,  say,  three  levels  of  operation:  Level  I  for  a  decision  maker 
relatively  unfamiliar  v;ith  decision  theory  and  with  the  operation  of 
CORTEX,  Level  II  for  a  decision  maker  moderately  experienced  in  the 
operation  of  CORTEX,  and  Level  III  for  a  decision  maker  highly  experienced 
in  both  decision  theory  and  the  use  of  CORTEX.  The  decision  maker  could 
then  specify  the  desired  level  immediately  after  calling  CORTEX. 

Lliile  CORTEX  must  be  used  in  order  to  be  considered  a  success,  it 
is  interesting  to  consider  whether  or  not  frequency  of  usage  is  an 
unambiguous  measure  of  effectiveness.  In  a  very  special  sense  it  is  not. 
Suppose  that  we  observe  a  decision  maker  who  utilizes  CORTEX  quite 
extensively  for  the  first  year  or  so  and  then  uses  it  less  and  less.  He 
may  be  becoming  less  satisfied  with  the  system  or  he  may  be  becoming  more 
and  more  convinced  that  it  is  not  really  helping  him  in  his  job.  Suppose 
that  we  take  a  closer  look  and  find  that  this  decision  maker  is  confining 
his  use  of  CORTEX  to  the  analysis  of  those  decision  problems  which  require 
only  the  most  complex  and  least  intuitively  comprehensible  algorithms. 

One  possible  cause  of  this  behavior  on  the  part  of  the  decision  maker  just 
miglit  be  that  through  using  CORTEX  he  has  learned  the  concepts  of  decision 
theory  and  the  operation  of  the  algorithms  so  v/ell  that  he  know  longer  needs 
to  use  the  system  to  arrive  at  a  satisfactory  understanding  of  and  solution 
to  many  of  his  decision  problems.  Thus,  the  decision  maker  has  internalized 
the  operation  of  CORTEX  and  can  now  make  many  decisions,  certainly  at  less 
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cost  and  more  rapidly,  than  the  computer-based  system  v;hich  is  now  partially 
outmoded  and  no  longer  needed. 

This  line  of  reasoning  suggests  that  an  effective  system  for  aiding 
decision  making  may  prove  to  be  an  effective  system  for  teaching  intuitive 
decision  making.  Though  the  mechanism  by  whicli  individuals  through  extensive 
experience  with  a  mathematical  system  gain  a  profound  appreciation  of  the 
behavior  of  the  system  is  not  well-understood,  it  certainly  exists.  Further, 
it  seems  reasonable  to  expect  that  this  internalization  of  mathematics  is 
more  likely  to  occur  for  some  processes  tiian  for  other  processes,  probably 
for  the  ^'simpler”  processes.  Since  CORTEX  is  not  conceived  as  a  static 
system  but  ratlier  as  one  which  evolves  to  incorporate  the  future  advances  in 
decision  theory  and  thus  will  include  more  and  more  ’’complicatecr'  processes, 
it  is  unlikely  that  a  decision  maker  through  internalization  V7ill  ever 
achieve  complete  independence  from  CORTEX.  However,  the  internalization  by 
some  decision  makers  of  some  of  the  processes  provided  by  CORTEX  should 
not  be  surprising  and  should  be  considered  a  thoroughly  desirable  consequence 
of  the  use  of  CORTEX. 
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by  significantly  reducing  the  special  knowledge  required  of  the  decision  maker. 
Thus,  the  decision  maker  needs  no  knowledge  of  computer  programming  and  a  min¬ 
imal  knowledge  of  decision  theory  and  mathematics  in  order  to  begin  using  the 
system  in  his  day-to-day  decision  making  activities. 
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